Performance of oxygenation indices and risk scores to predict invasive mechanical ventilation and mortality in COVID-19

Background Information on the performance of oxygenation indices (OIs) and risk scores in patients requiring invasive mechanical ventilation (IMV) is limited. We determine the performance of the OIs and risk scores in hospitalized patients with COVID-19 to predict the requirement of IMV and death at 28 days after admission. Methods A retrospective study of diagnostic tests in patients admitted to the emergency department, hospitalization, and intensive care unit diagnosed with COVID-19. The receiver operating characteristic curve (ROC-curve) were built with the OIs and risk scores to predict IMV and mortality. Results A total of 1402 subjects entered the final analysis, of whom 19.5% (274/1402) received IMV and 23.0% (323/1402) died at 28 days. The ROC-curve of the delta PaO2/FiO2 ratio for the requirement of IMV and mortality at 28-day was 0.589 (95% CI: 0.546–0.632) and 0.567 (95% CI: 0.526–0.608), respectively. PaO2/FiO2 ≤ 300 shows a ROC curve of 0.669 (95% CI: 0.628–0.711) to predict IMV. PaO2/FiO2 ≤ 300 and 4 C mortality score in mortality at 28 days showed an ROC-curve of 0.624 (95% CI: 0.582–0.667) and 0.706 (95% CI: 0.669–0.742), respectively. Conclusion PaO2/FiO2 ≤ 300, 4 C mortality score ≥ 8, SOFA score ≥ 4 y SaO2/FiO2 ≤ 300 were weak predictors of the IMV requirement from admission, and 4 C mortality score ≥ 8 was weak predictors of the mortality from admission in patients with pulmonary involvement by COVID-19. Supplementary Information The online version contains supplementary material available at 10.1186/s12890-023-02807-8.


Introduction
Infection caused by severe acute respiratory syndrome coronavirus type 2 (SARSCoV-2) is responsible for the rapid global spread and current pandemic of coronavirus disease 2019 (COVID- 19), which continues to be a threat to public health due to the persistence of reported cases, 6.5 million deaths and the physical sequelae associated with the disease [1,2].80% or more of COVID-19 cases are asymptomatic or mild, however, less than 15% patients with genetic predisposition, comorbidities, or advanced age may develop severe or critical illness with multiple organ failure and acute respiratory distress syndrome, complications causing a high expenditure on medical care resources and a high mortality rate [2,3].
The main clinical finding in patients with moderate to severe COVID-19 is decreased oxygen saturation values due to diffuse alveolar damage present in acute respiratory distress syndrome and an uncontrolled inflammatory state, [4,5] generating an imbalance of gas exchange due to destruction of the lung parenchyma, hypercatabolic state and a greater requirement of oxygen pressures for its adequate diffusion through the alveolar capillary membrane [5,6].Therefore, patients with severe COVID-19 may require early invasive mechanical ventilation, continuous positive airway pressure, or high-flow nasal cannula, together with strict follow-up consisting of clinical and laboratory tests, including oxygenation indices (OIs) [4,7,8].
The use of OIs as predictors of clinical evolution in patients with COVID-19 has already been studied by several authors; Sinatti et al., [9] conducted a cohort study with 150 patients and found that arterial pressure of oxygen/fraction of inspired oxygen (PaO2/FiO2 ratio) can be considered a reliable prognostic biomarker to differentiate severe from mild disease with yield based on the receiver operating characteristic curve (ROC-curve) of 0.838.Xie J et al., [10] showed in 140 patients with pneumonia associated with COVID-19 that supplemental oxygen with a peripheral oxygen saturation (SpO2) lower than 90% was related to sample in more than 60% of the patients, and that an increase in SpO2 improved survival by 8% (HR: 0.92; 95% CI: 0.91 to 0.94; p < 0.001).These results suggest that the evaluation of baseline OI values and their change values in the first hours of clinical follow-up can guide the evolution and prognosis of these patients.The other hand, ROX index ('Respiratory rate-OXygenation') is the ratio between peripheral blood oxygen saturation (SpO2) to fraction of inspired oxygen (FiO2) and respiratory rate, which has shown promising performance in successful prognosis of invasive and noninvasive oxygen therapy [11][12][13].
OIs are useful markers for the prediction of complications and mortality in patients with lung involvement, including subjects with COVID-19 [4,9,10].Currently, information on the performance and discriminatory capacity of OIs in patients requiring invasive ventilatory support and mortality are limited, generating significant variability in clinical practice [8,14].Based on this, this study aims to determine the performance of OIs, delta of partial pressure of arterial oxygen to fraction of inspired oxygen (PaO2/FiO2) ratio, delta SpO2/FiO2 ratio, ROX index, Sequential Organ Failure Assessment (SOFA) score, 4 C mortality score and Charlson index in hospitalized patients with COVID-19 to predict invasive mechanical ventilation (IMV) and mortality at 28 days after admission.

Study design
A retrospective study of diagnostic tests was carried out in patients admitted to the emergency department, hospitalized, and admitted to the intensive care unit (ICU) of the Clínica Universidad de La Sabana between March 2020 and March 2022.This study followed STARD guidelines for reporting diagnostic or prognostic accuracy Supplementary Table 1.

Eligibility criteria
The patients included in this study were over 18 years of age and hospitalized for COVID-19 pneumonia for more than 72 h, confirmed by a positive reverse transcription polymerase chain reaction (RT-PCR) test for SARS-CoV-2, obtained with a nasopharyngeal swab/tracheal aspirate or sputum sample.Patients with a history of congenital heart disease leading to chronic hypoxemia and/or home oxygen therapy for these conditions, those requiring mechanical ventilation within the first 6 h of emergency room admission, and those with a single measurement of arterial blood gases that did not allow for clinical follow-up were excluded.Subjects were selected by simple random sampling from the list of patients seen during the study period.

Analyzed variables
The variables described were age, sex, days since symptoms start, comorbidities (Charlson Index) and eversmoking or currently smoking tobacco products based on self-reported, vital signs, state of consciousness, complete blood count, ROX index, SOFA score, 4 C mortality score, arterial blood gas, bicarbonate, base excess, arterial oxygen saturation, lactate dehydrogenase, albumin, creatinine, blood ureic nitrogen, lactate dehydrogenase, glutamic-oxaloacetic transaminase, gamma-glutamyl transferase, troponin, procalcitonin, C-Reactive protein, total bilirubin, ferritin, creatine phosphokinase and chest computer tomographic, these data was obtained from medical records at the time of admission to the hospital.SpO2/FiO2 ratio ≤ 350, PaO2/FiO2 ratio ≤ 300, ROX index ≥ 4.88 were calculated from the first 6 h of admission, between 6 and 12 h, 12 to 24 h, and over 24 h [15][16][17].Each of the indices were calculated from oximetry records and arterial blood gas measurements.The duration of hospitalization, vasopressor support in the ICU, IMV requirement (7, 14 and 28 days), and death (7, 14 and 28 days) were evaluated.
The cut-off points used for each risk score were ≥ 4.88 for the ROX index, > 4 for the SOFA score and ≥ 8 for the 4 C mortality score (age, respiratory rate, oxygen saturation, blood urea nitrogen and C-Reactive protein) at 12 h as an indicator of failure [11,18,19].The variables of each risk score are described in Supplementary file 2.

Sample size
To estimate the sample size, data from the study carried out by Alberdí et al., [20] that evaluated SpO2/FiO2 ratio and ROX index, where a sensitivity of 26.8%, specificity of 89.5% for these indices and an expected mortality of 35.3% are reported.For a confidence level of 95% and a precision of 10%, a minimum of 856 patients is required [21].

Statistical analysis
Data was fully collected and compiled using a secure server (Research Electronic Data Capture, REDCap software) and later analyzed in the SPSS (Statistical Package for the Social Sciences, Chicago, IL) for Windows version 25 [22,23].The data were obtained from the medical records of the patients included in the study, and the collection of information was carried out by at least two researchers to reduce the risk of errors in data entry.Quantitative variables were summarized in means and standard deviations if their distribution was normal, or median and interquartile range if their distribution was not normal.The qualitative variables were summarized in frequencies and percentages.A bivariate analysis was performed comparing the sociodemographic variables, comorbidities, laboratory tests, OIs and risk scores with the live and dead outcomes because it provides a good characterization of this population.Differences (delta) were calculated between the SpO2/FiO2 ratio, the PaO2/ FiO2 ratio on admission, with those obtained in the first 6 h, between 6 and 12 h, between 12 and 24 h, and over 24 h.For missing data, a weighted median was performed for quantitative variables and logistic regression for qualitative variables [24].
OI (PaO2 mmHg < = 60, SaO2% <= 90, PaO2/ FiO2 ≤ 300, SpO2/FiO2 ≤ 350, ΔPaO2/FiO2 ratio, ΔSaO2/ FiO2 ratio) of the first 6 h of admission, Charlson index, ROX index of the first 6 h of admission, 4 C mortality score, and SOFA score were calculated to construct the ROC curve with the results of IMV requirement and mortality at 7, 14, and 28 days.The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR-) with their respective 95% confidence intervals were calculated.The cut-off points for the calculated deltas were obtained through the Youden index.A comparison was made between the different ROCcurves obtained, using the DeLong test [25].A p value adjusted by Bonferroni is considered for the comparison of the different OI and risk scores less than < 0.006.

Ethical considerations
This study was approved by the Ethics Committee of the Clínica Universidad de La Sabana (approval number: 20,220,602), considering it as risk-free research according to resolution 8430 of 1993, and respecting the protection of personal data according to the habeas data law 1266 of 2008.

Patient and public involvement
Patients were not involved in the development of the research question, design, recruitment, or intervention burden assessed; no patient advisors were required, and data were analyzed anonymously.The results will be disseminated to the scientific community in academic writing.

General characteristics of the population, chromobilities and symptoms
A total of 1402 subjects entered the final analysis, of whom 19.5% (274/1402) received IMV and 23.0% (323/1402) died at 28 days.The 7-day mortality rate was 6.7% (95/1402), and it increased to 15.4% (216/1402) at 14 days.In Fig. 1 the entry flow of subjects to the study is shown.In the general population, the mean age was 59.9 years (SD 16.19), the male sex represented 63.3% (888/1402) and the duration from the onset of symptoms was 7.3 days (SD: 13.26).In the deceased population, 46.4% (150/323) had systemic arterial hypertension compared to 34.5% (372/1079) of the surviving patients (p < 0.001).Cough and crackles occurred in 47.7% (154/323) and 38.4% (124/323) of the patients who died, respectively.The general characteristics are described in Table 1.

Laboratory tests and diagnostic images
The mean pH was 7.43 (SD: 0.07) in the patients who had a fatal outcome versus 7.45 (SD: 0.06) in the survivors (p < 0.001).In the population that died, the average C-reactive protein was 146.4 (SD: 115.74) compared to 121.5 (SD: 107.42) in living patients (p < 0.001).Laboratory tests and diagnostic images are described in Table 2.
The ROC-curve of the OIs and the risk scores to predict IMV and mortality are shown in the Fig. 2 and Fig. 3.
Analysis with the De Long test showed that at 7, 14 and 28 days the ROC-curves with the best performance for IMV were PaO2/FiO2, 4 C mortality score, SOFA score and SaO2/FiO2, the ROC curves of the rest of the OI evaluated were lower (p < 0.001).At 7 days, no statistically significant differences were found for multiple comparisons between the OI and risk scores assessed for death (p = 0.043).At 14 days, ROC-curves with the best behavior for death were the 4 C mortality score and the Charlson index, the ROC-curves of the rest of the OI and SOFA scores evaluated were lower (p < 0.001).At 28 days, the ROC-curve with the best behavior for death was 4 C mortality score compared to the ROC-curves of the rest of the OI and risk scores evaluated (p < 0.001).Notes: S: Sensibility, Sp: specificity, PPV: positive predictive value, NPV: negative predictive value, LR +: positive likelihood ratio, LR-: negative likelihood ratio, CI: confidence intervals, ROC: receiver operating characteristic curve, PaO2: arterial oxygen pressure, SaO2: arterial oxygen saturation, Δ: delta, PaO2/FiO2 ratio: arterial oxygen pressure/inspired fraction of oxygen, SaO2/FiO2 ratio: arterial oxygen saturation in relation to the inspired oxygen fraction, ROX: Respiratory rate-OXygenation index, SOFA: Sequential Organ Failure Assessment

Discussion
The present study determined the performance of the change in OIs and risk scores in a retrospective study of diagnostic tests of 1402 hospitalized patients with COVID-19, finding that the PaO2/FiO2, 4 C mortality score, SOFA score y SaO2/FiO2 were weak predictors of the IMV requirement from admission, and 4 C mortality score was weak predictors of the mortality from admission.The delta PaO2/FiO2 and the delta SaO2/FiO2 showed inferior performance for IMV and death compared to the other tools evaluated.Characteristics such as age, pathological history, and clinical manifestations occurred more frequently among patients who died from SARS-CoV-2.Our findings suggest that the predictive capacity for the requirement of IMV and mortality is limited.This is attributed to the fact that ROC-curves were independently calculated for each of the OIs and risk indices, which primarily focus on describing respiratory compromise and the reduction in gas exchange across the alveolar-capillary membrane [2,3,9,10].Therefore, conducting integrated assessments that consider both OIs and risk indices is essential to estimate damage across various organs or systems, as commonly observed in severe pneumonia cases caused by COVID-19.PaO2/FiO2 reflects the severity of hypoxemia and given its performance, has been correlated with prognosis and hospital mortality in patients with acute respiratory failure due to COVID-19 [26][27][28].Direct blood sample collection for gasometric analysis it is the main reference for evaluating the oxygenation status of patients with respiratory diseases, [9] and its baseline value is the one that offers the most information in the evaluation of the patient [9,27,28].In our study, the PaO2/FiO2 ratio showed a weak discriminatory capacity for IMV or death.In addition, the continuous evaluation of oxygenation status through gas analysis constitutes an expensive invasive procedure and was not available at the different levels of care, requiring the continuous study of other measurements that use pulse oximetry to obtain indices such as the SpO2/FiO2 ratio and ROX index, which can reflect the state of hypoxemia in a non-invasive way [29].
Knight et al., [18] developed and validated a 4 C mortality score (Coronavirus Clinical Characterization Consortium) of risk for the prediction of mortality in a derivation cohort with 35.463 patients and validation with 22.361 patients hospitalized for COVID-19.The variables of age, gender, number of comorbidities, respiratory rate, SpO2, state of consciousness, urea nitrogen, and C Reactive protein were included in the score; obtaining a scale of 0 to 21 points and a performance of 0.79 (95% CI: 0.78-0.79) in the derivation cohort and 0.77 (95% CI: 0.76-0.77) in the validation cohort for mortality, similar findings in our study where greater comorbidity measured by Charlson and greater severity by 4 C mortality score were related to greater mortality in patients with COVID-19 [18,30,31].
Cattazzo et al., [32] analyzed the efficiency of the ROX index and the SaO2/FiO2 ratio compared to PaO2/FiO2 ratio for the prediction of death or IMV requirement in 456 patients hospitalized in areas other than the ICU due to COVID-19.The results showed an ROC-curve of 0.67 (95% CI: 0.62-0.73;p < 0.001) for PaO2/FiO2 ratio, 0.69 (95% CI: 0.63-0.74;p < 0.001) for the ROX index and 0.66 (95% CI: 0.60-0.72;p < 0.001) for SpO2/FiO2 ratio; similar situation to what we find where the basal values of each of these indices have a weak discriminatory capacity for IMV or death.Baek et al., [33] in patients with COVID-19 and supplemental oxygen with a highflow cannula found that the change in the ROX index and SaO2/FiO2 ratio was related to early IMV; findings that were corroborated with our results where a drop in SaO2/FiO2 ratio of 150 points was associated with a greater requirement of IMV and death, the change in this variable being the one with the best efficiency for the prediction of complications in patients with SARS-CoV2 infection.
Grasselli et al., [34] described demographic characteristics, comorbidities, and treatments of a cohort of 3,988 critically ill patients with SARS-CoV-2 infection.Hypertension, hypercholesterolemia, heart disease, diabetes, chronic obstructive pulmonary disease, and chronic kidney disease were associated with increased mortality.Our results confirm that deaths from COVID-19 were higher in patients with multiple comorbidities, a consequence of a pre-existing inflammatory state in chronic diseases and greater susceptibility to a cellular immune response and humoral activation, predominantly of tumor necrosis factor and interleukin 6 [35,36].This was possibly related to what was observed in most clinical scenarios as described during the pandemic period, in which the presence of other respiratory diseases directly influences the loss of alveolar reserve, promoting greater pulmonary compromise secondary to viral infection [37,38].

Limitations
As it was a retrospective study based on medical records, this study may give rise to selection and information biases; However, we implemented measures to minimize bias, such as training the personnel in charge of collecting medical data and constructing the manuscript based on the checklist of items that should be included in the reports of retrospective diagnostic test study Supplementary Table 1.Similarly, being a single center study may limit the extrapolation of the results, despite this, there was a sufficient sample size to support them.Unlike the ROX index and the 4 C mortality score, the SOFA score has an important limitation in its daily application, since it depends on variables that are not always available in medical care centers or hospitals, especially in countries with limited resources, as in our study population [11,18,19].
The altitude above sea level where the care center was located can be considered a limiting factor, since exceeding 2.500 m above sea level can alter oxygenation values, as described in previous studies [13,39].However, there are large numbers of the world's population residing at altitudes to which these findings could be applied.The lack of a uniform take in the evaluation of arterial gasses during the evolution of patients with SARS-CoV2 can cause confusion throughout the follow-up, however, each of the analyzes at different periods of time carried out in the study consistently show differences in the OIs evaluated.We consider it pertinent to carry out future studies that corroborate our findings.

Conclusions
PaO2/FiO2 ≤ 300, 4 C mortality score ≥ 8, SOFA score ≥ 4 y SaO2/FiO2 ≤ 300 were weak predictors of the IMV requirement from admission, and 4 C mortality score ≥ 8 was weak predictors of the mortality from admission in patients with pulmonary involvement by COVID-19.Age, pathological history, and clinical manifestations occurred more frequently among patients who died from SARS-CoV-2.

Table 1
Demographic characteristics, medical history, and risk scores Notes: sd: standard deviation, ROX: Respiratory rate-OXygenation index, SOFA: Sequential Organ Failure Assessment, *: measurement at hospital admission

Table 3
Performance of invasive mechanical ventilation and mortality at 28-day